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SUMMARY1  PAGE 


THE  PR03LEM 

The  tests  currently  available  for  the  selection  of  men  to  be  trained  as  naval 
aviators  leave  room  for  improvement.  This  is  a  study  to  determine  whether  any  of 
several  experimental  tests  might  add  significantly  to  the  validity  of  the  present  battery. 

FINDINGS 

A  short-form  of  the  Instrument  Comprehension  test  and  the  more  valid  items  of  the 
Background  test  appear  to  merit  revalidation  under  actual  selection  conditions.  One 
or  two  of  the  scores  from  the  Prestige/Sscurity  scale  may  add  to  the  validity  of  second¬ 
ary  selection  formulae,  i.e.  those  used  to  estimate  the  probable  success  of  students  who 
are  already  in  training. 


INTRODUCTION 


This  report  describes  efforts  to  develop  several  new  tests  that  might  add  to  the 
validity  of  the  battery  used  to  select  naval  aviators.  The  current  battery  includes  a 
general  intelligence  test,  the  Aviation  Qualification  Test  (AQT),  which,  while  mainly 
verbal-numerical,  also  contains  some  items  that  measure  clerical  accuracy.  Scores  on 
this  test  relate  to  performance  in  academic  subjects,  usually  with  correlations  of  about 
.60  in  unrestricted  samples.  The  balance  of  the  battery  includes  two  tests,  the  Mech¬ 
anical  Comprehension  Test  (MCT)  and  the  Spatial  Apperception  Test  (SAT),  that  discrim 
inate  between  potential  flight  failures  and  successes,  and  a  third  test,  the  Biographical 
Inventory  (Bl),  that  has  a  useful  predictive  relationship  to  voluntary  withdrawal  from 
the  flight  training  program.  Scores  from  these  latter  three  tests  make  up  a  composite 
"Flight  Aptitude  Rating"  (FAR)  which  is  expressed  in  stanine  scores.  In  an  unrestricted 
sample  these  scores  usually  have  a  correlation  of  .40  to  .50  (biserial)  with  a  dichoto¬ 
mous,  completed/dropped  criterion. 

Figure  I  shows  the  relations1, >d  of  the  FAR  battery  scores  to  success  in  flight  train¬ 
ing.  It  can  be  seen  that  even  ,r  th^  two  highest  stanine  groups,  about  15  per  cent  of 
the  selectees  failed  to  complete  the  program.  It  was  reasoned  that  if  there  were  flight 
training  failures  among  this  15  per  ~ent,  an  examination  of  the  causes  of  their  failures 
might  reveal  areas  of  flying  ability  not  now  covered  by  the  tests. 

Twenty  such  persons  were  found,  and  the  comments  made  about  their  flying  by  the 
flight  instructors  were  examined.  Two  comments  occurred  sufficiently  often  to  be  pro¬ 
vocative:  The  first  described  a  per  .stent  pattern  of  landing  too  high  or  too  low,  an 
apparent  inability  to  judge  the  height  of  the  plane  in  relation  to  the  runway;  the  second 
dealt  with  the  students  being  unable  to  proceed  correctly  through  the  sequence  of 
actions  required  to  complete  a  maneuver. 


Relationship  of  FAR  Scores  to  Success  in  Flight  Training 
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EXPERIMENTAL  TESTS 


Two  tests  hopefully  Intended  to  measure  these  abilities  were  devised,  "Altitude 
Judgment"  and  the  "Maneuvers  Test."  For  the  Altitude  Judgment  test  a  large  number  of 
35-mm  photographs  of  the  runway  were  taken  through  the  windshield  as  a  plane  was 
coming  in  fora  landing.  They  were  taken  at  various  points  in  the  descent  and  with  the 
plane  in  varying  aititudes  and  positions  relative  to  the  landing  strip.  From  the  finished 
slides  reproducible  drawings  were  made  by  projecting  the  slides  on  a  glass  screen  and 
tracing  them  on  translucent  paper.  A  selection  of  these  was  made  for  the  test,  a  portion 
of  which  is  shown  in  Appendix  A. 

The  "Maneuvers  Test"  described  the  operating  characteristics  of  a  hypothetical, 
simplified  airplane.  It  then  asked  the  subject  to  specify  the  sequence  of  throttle,  rpm, 
and  stick  positions  necessary  to  complete  a  series  of  maneuvers.  These  included  "take¬ 
off,"  "climbing  turn,"  "figure  eight,"  "Immelman,"  "power-on  glide,"  "field  entry," 
and  "landing."  The  correct  answer  might  require  as  many  as  twelve  steps  which  had  to 
be  in  correct  sequence.  While  a  trial  run  of  this  test  indicated  some  validity,  the  test 
was  too  difficult  and  the  original  scoring  methods  were  unsatisfactory  for  general  use. 
Efforts  are  current  to  develop  a  more  practical  version  of  the  test. 

A  third  test  was  developed  by  choosing  from  the  sixty-item  Instrument  Comprehen¬ 
sion  Test  used  in  the  selection  of  Air  Force  Pilots  the  twenty  items  with  the  largest 
correlations  with  total  test  score.  A  sample  item  is  shown  in  Appendix  B. 

These  three  »ests  were  given  preliminary  evaluations  by  administering  them  to  a 
mixed  input  of  aviation  trainees  and  obtaining  their  point  biserial  correlations  with  a 
criterion  of  pass/fail  in  basic  training.  Table  I  shows  the  results.  The  obtained  co¬ 
efficients,  while  not  very  exciting,  were  significant  at  the  .01  level  and  suggested 
further  investigation. 


Table  I 

Preliminary  Evaluation  of  Three  Experimental  Tests 


Test 

No.  Succeeded 

No.  Failed 

Biserial  * 

Altitude  Judgment 

480 

78 

.161 

Maneuvers 

310 

37 

.169 

Instrument  Comprehension 

335 

61 

.140 

Two  other  experimental  tests  were  included  in  the  evaluation  covered  by  this  report. 
One  was  called  the  "Background"  test  (1).  This  is  a  twenty-item  scale  used  to  estimate 
the  cultural  (or  socioeconomic)  level  of  the  homo  at  the  time  that  the  respondent  was  in 
high  school.  The  remaining  experimental  measure  is  shown  in  Appendix  C  and  derives 
from  a  scale  originated  by  P.  Nelson  (2).  The  scale  lists  seventeen  occupations  as 
they  were  ranked  in  prestige,  and  then  in  security  by  aviation  trainees.  The  subject  is 
asked  to  place  the  naval  aviator  and  the  naval  flight  officer  where  he  feels  they  belong 
in  these  rankings.  Four  scores  were  obtained;  the  Prestige  and  Security  scores  were  the 
numbers  of  the  levels  at  which  the  subject  placed  the  naval  aviator.  The  Prestige 
Difference  and  the  Security  Difference  scores  were  the  differences  between  the  ranks 
that  the  subject  assigned  to  aviators  and  to  NFO's. 

PROCEDURE 

The  experimental  tests  were  administered  to  aviation  students  during  their  first  week 
at  Pensacola.  Two  years  later  the  data  were  divided  into  rbe  scores  of  those  who  com¬ 
pleted  training  and  of  those  who  did  not.  The  number  of  students  was  large  enough  also 
to  permit  division  into  Aviation  Officer  Candidates,  who  are  college  graduates,  and 
civilian  procured  naval  cadets,  who  normally  have  two  years  of  college.  Selection 
variables  and  a  mathematics  qualifying  examination,  also  given  during  the  first  week, 
were  included  in  the  analysis. 

RESULTS 

The  correlations  of  the  variables  with  each  other  ond  with  a  complete/attrite  cri¬ 
terion  were  computed.  The  Wherry-Dool ittle  procedure  was  used  to  identify  the  best 
combination  of  variables  with  which  to  predict  attrition,  and  the  appropriate  weights 
were  determined.  Table  II  shows  the  results  for  AOC's  during  Pre-Flight  training. 

The  Prestige  Difference  score  and  the  Instrument  Comprehension  score  were  the 
second  and  third  variables  chosen.  The  Background  and  Security  Difference  scores 
also  added  to  the  multiple  validity;  however,  the  amounts  they  added  were  of  little 
practical  importance. 

Table  III  shows  the  prediction  formula  for  naval  cadets  during  the  same  stage  of 
training. 

This  formula  differs  somewhat  from  the  formula  for  AOC's  in  that  the  Security 
Score  replaced  the  Prestige  Difference,  and  the  SAT  replaced  the  Instrument  Compre¬ 
hension  Test. 

Table;  IV  and  V  show  comparable  data  taken  at  the  end  of  Pre-Flight  School  and 
including  Pre-Flight  grades  and  ratings  in  the  matrix. 
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Toble  II 


Prediction  Formula  for  AOCV  During  Pre-Flight  Training 


Variable 

Cumulative 
Shrunken  !<** 

Raw  Score 

Weight 

MCT 

.2390 

.016 

Prestige  Difference 

.2854 

.038 

Instrument  Comprehension 

.2925 

.010 

Bl 

.2956 

.003 

Background 

.2958 

-.004 

Security  Difference 

.2961 

-.012 

*N  =  407 

**R,  point  biserial 

Variables  not  selected:  Age,  1 

Education,  AQT, 

SAT,  Incoming  Math,  Altitude 

Judgment,  Prestige  Score,  Sec 

urify  Score.  (The  matrices  on  which  Tables  II  through 

V  are  based  are  shown  in  Appendix  D.) 

Prediction  Formul 

Table  III 

a  for  Cadets*  Dur 

ing  Pre-Flight  Training 

Variable 

Cumulative 

Raw  Score 

Shrunken  R 

Weight 

MCT 

.1802 

.011 

Bl 

.2331 

.006 

Security  Score 

.2576 

.018 

Age 

.2721 

-.002 

Background 

.2832 

-.006 

SAT 

.2869 

.006 

AQT 

.2935 

-.003 

*  N  =  379 


Variables  not  selected:  Education,  Incoming  Math,  Altitude  Judgment-  Instrument 
Comprehension,  Prestige  Score,  Prestige  Difference,  Security  Difference. 
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Table  IV 


Prediction  Formula  for  AOC's*  of  End  of  Pre-Flight  Troining 


Variable 

Cumulative 
Shrunken  R 

Raw  Score 

Weight 

Power  Plants 

.2353 

.007 

Peer  Rating 

.2844 

.009 

MCT 

.3174 

.013 

Prestige  Difference 

.3539 

.039 

Bl 

.3560 

.002 

AQT 

.3570 

-.003 

Instrument  Comprehension 

.3578 

.007 

Security  Difference 

.3587 

-.012 

*N  =405 

Variables  not  selected:  Age,  Leadership,  Navigation, 

Incomina  M?th,  Prestige  Score, 

Education,  Physiology,  Pri 
Orientation,  Study  Skills, 

nciples  of  FI  ight.  Background,  Security  Score,  SAT,  Naval 
Altitude  Judgment. 

Table  V 

Prediction  Formula  for  NavCads*  at  End  of  Pre-Flight  Troining 

Variable 

Cumulative 

Raw  Score 

Shrunken  R 

Weight 

Power  Plants 

.2548 

.005 

Navigation 

.2870 

.008 

Bl 

.3115 

.005 

Peer  Rating 

.3301 

.006 

Education 

.3442 

-.039 

Background 

.3539 

-  C06 

Security  Score 

.3630 

.014 

Age 

.3691 

-.002 

AQT 

.3762 

-.005 

MCT 

.3830 

.006 

Physiology 

.3843 

.003 

*N  =  377 


Variables  not  selected:  SAT,  Principles  of  FI ight.  Altitude  Judgment,  Prestige 
Difference,  Leadership,  Stud/  Skills,  Instrument  Comprehens:on,  Security  Difference, 
Naval  Orientation,  Incoming  Math,  Prestige  Score. 
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4.1 


DISCUSSION 

First  it  should  be  noted  that  the  multiple  correlations  shown  in  Tables  II  through  V 
are  from  samples  that  have  been  severely  restricted  in  range  on  the  selection  variables. 
This  is  so  because  individuals  with  low  scores  on  the  Flight  Aptitude  Rating  and  on  the 
AQT  were  not  accepted  into  the  program.  One  effect  of  this  is  to  limit  the  size  of  the 
correlations  that  can  be  obtained  using  these  variables.  But  another  is  to  increase  the 
likelihood  that  other  variables,  not  deliberately  restricted  by  the  selection  process, 
will  enter  into  the  obtained  multiple  correlations.  It  cannot  be  assumed,  therefore, 
that  those  experimental  measures  which  add  to  the  multiple  correlations  here  would  also 
add  to  them  in  the  initial  selection  situation,  where  the  variance  of  the  present 
selection  tests  is  still  unrestricted. 

Table  VI  shows  which  experimental  tests  contributed  to  each  prediction  formula. 
The  Altitude  Judgment  test  did  not  get  into  any  of  the  four  multiples.  Thus  it  can  be 
concluded  that  the  validity  shown  for  it  in  the  preliminary  study  (Table  I)  duplicates 
variance  that  is  present  in  other  tests.  From  the  intercorrelation  matrices  (Appendix  D) 
it  can  be  seen  that  these  tests  are  probably  the  MCT  and  the  Instrument  Comprehension 
test. 


Table  VI 


Contributions  of  Experimental  Tests  to  Multiple  Correlations 


Experimental  Test 

Variable 

AOC 

During 

Pre-Flight 

NavCod 

During 

Pre-Flight 

AOC 

After 

Pre-Flight 

NavCad 

After 

Pre-Flight 

Altitude  Judgment 

— 

«... 

—  — 

Instrument  Comprehension 

>/ 

— 

y 

— 

Background 

0 

— 

0 

Prestige  Score 

— 

— 

— 

... 

Prestige  Difference 

y 

y 

Security  Score 

— 

y 

✓ 

Security  Difference 

0 

— 

0 

— 

‘Checkmarks  in  circles  indicate  negative  Beta  weights. 
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The  Instrument  Comprehension  Test  entered  into  the  formulae  involving  AOC's  but 
not  those  involving  NavCads.  Given  the  low  correlations  and  small  increments  with 
which  we  are  dealing,  this  can  probably  be  ascribed  to  chance.  One  should  investigate 
this  test  further  in  the  primary  selection  situation.  The  same  can  be  said  of  the  Back¬ 
ground  Test  which  added  somewhat  to  three  of  the  multiples.  It  exhibited  negative 
Beta  weights,  higher  socioeconomic  level  being  associated  with  higher  attrition  rates. 
The  more  valid  items  from  this  test  could  very  easily  be  added  to  the  current  Biograph¬ 
ical  Inventory  and  revalidated  under  selection  conditions. 

The  Prestige/Security  Scale  presents  a  different  case,  however.  Three  of  the  four 
scores  had  some  validity,  but  completion  of  the  tests  appears  to  require  more  knowledge 
of  the  pilot/NFO  career  options  than  men  in  the  primary  selection  situation  would  be 
likely  to  have.  Thus  this  test  should  probably  best  be  given  during  test  time  that  is 
available  during  the  fifth  week  and  the  results  validated  when  new  secondary  selection 
(student  prediction)  formulae  are  developed. 

CONCLUSIONS 

The  short-form  Instrument  Comprehension  test  should  be  validated  with  the  present 
selection  battery. 

The  valid  items  from  the  Background  Test  should  be  incorporated  into  the  Biograph¬ 
ical  Inventory  and  revalidated  under  selection  conditions. 

The  Pi  'stige/Securify  Scale  should  be  administered  during  the  fifth  week  of  train¬ 
ing  and  the  scores  included  in  the  next  developmental  analysis  for  student  prediction 
formulae. 
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APPENDIX  A 

Partial  Instructions  and  Sample  Items  from  the  Altitude  Judgment  Test 


This  is  a  test  of  your  ability  to  fudge  the  altitude  of  an  aircraft  from  the  cockpit.  The 
pictures  used  in  the  test  are  drawings  made  from  photographs  taken  during  actual  land¬ 
ings.  Four  of  the  pictures  are  identified  by  the  letters  W,  X,  Y,  2.  Twelve  are 
numbered.  You  are  to  compare  each  of  the  lettered  pictures  with  each  of  the  number¬ 
ed  pictures . 

IF  PLANE  SEEMS  HIGHER  IN  IF  PLANE  SEEMS  HIGHER  IN 

LETTERED  PICTURE  -  MARK  A  NUMBERED  PICTURE  -  MARK  B 


X  2 


Y 
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APPENDIX  B 

Partial  Instructions  for  Instrument  Comprehension  Test 


The  compass  Indicates  the  direction  In  which  the  plone  it  headed.  Not#  the*#  somple  reoding*: 


HEADED  NORTH  HEADED  WEST 

The  artificial  horizon  has  two  functions, 
indicates  the  amount  and  direction  of  bonk  of  the  plane. 

When  climbing,  the  horizon  lines  appear  below  the  small  plone 
os  in  dial  I.  When  diving,  the  horizon  lines  appear  above 
the  small  plane  as  in  dial  2.  When  flying  level,  the  horizon 
lines  ore  even  with  the  unoll  plane. 


The  arrowhead  in  the  dial  registers  the  number  of  degrees  of 
bank .  A  left  bank  is  read  in  number  of  degrees  to  the  right 
of  zero  at  in  dial  3.  A  right  bank  is  read  in  number  of 
degrees  to  the  left  of  zero  as  in  diol  4. 


When  the  plon#  is  both  banked  and  climbing,  the  dial  appears  t 
at  in  dial  5.  When  the  plane  is  both  banked  and  diving, 
the  diol  appears  os  in  dial  6. 


HEADED  NORTHWEST  HEADED  NORTH  OF  N  W 


First ,  it  shows  whether  the  plane  is  climbing,  diving,  or  flying  level.  Second 


I.  CLIMBING  2.  09  IG 


3.  LEFT  BANK  4.  RIGHT  BANK 


».  LEFT  BANK,  6  RIGHT  BANK, 
CLIMBING  DIVING 


Now  examine  the  diol  readings  in  problem  8.  Then  select  the  correct  position  of  the  plone. 


PROBLEM  B 


ARTIFICIAL  COMPASS 
H0RI70N 


N'5^ 


A  B  C  D  E 


According  to  the  dials,  the  plone  is  banked  left,  flying  level,  and  is  headed  south.  (8)  is  the  correct  answer, 
because  at  position  (8)  the  plone  is  bonked  left,  flying  level,  ond  is  heoded  south. 

In  each  problem  choose  the  position  of  the  plane  that  is  correct  for  the  dial  reodings;  then  blacken  the  space  on 
the  answer  sheet  which  corresponds  to  the  answer  you  select.  If  you  are  not  sure  which  is  the  correct  answer, 
moke  the  best  guess  you  con.  Work  rapidly  and  carefully  until  you  <*€  tpfd  to  stop.  When  you  finish  one  page, 
go  on  fo  the  next. 

You  will  have  5  minutes  to  complete  the  20  items.  If  you  finish  before  time  is  colled,  you  may  go  back  over 
your  work  . 


DO  NOT  TURN  THIS  PAGE  UNTIL  TOLD  TO  DO  SO. 


APPENDIX  C 


NAME  CLASS  NO.  DATE 

TOT - FT*5T - mvm -  -  - 

AOC _ N/C _ M/C _ A! _ NAO _ Ol _ 

The  occupations  below  were  ranked  by  350  naval  aviation  trainees.  Note  that  the 
left-hand  list  is  ranked  from  top  to  bottom  according  to  "PRESTIGE,"  while  the  right- 
hand  list  is  ranked  in  terms  of  "CAREER  SECURITY."  Compare  the  prestige  of  the 
Naval  or  Marine  Aviator  with  that  of  the  occupations  in  the  left-hand  list  and  write  in 
the  letters  NA  at  that  point  in  the  list  where  you  feel  the  Naval  or  Marine  Aviator  fits. 
Do  the  same  for  the  Naval  Aviation  Officer  (non-pilot)-NAO.  Then,  in  the  right- 
hand  list  compare  the  NA  and  NAO  career  security  with  that  of  the  occupations  listed 
and  write  each  into  the  appropriate  space. 


PRESTIGE 


CAREER  SECURITY 


Physician 

17 

Physician 

Scientist 

16 

Engineer 

Minister 

b 

Scientist 

Lawyer 

14 

Minister 

Engineer 

13 

Banker 

College  Professor 

12 

College  Professor 

Banker 

U 

Lawyer 

Architect  TO  Architect 


Politician  9  News  Column ist 


Businessman 

6 

Accountant 

News  Columnist 

7 

Businessman 

feodio  -  TV  Announcer 

6 

Personnel  Director 

Personnel  birector 

5 

Farmer 

Sales  Promotion 

4 

High  School  Coach 

High  School  Coach 

3 

ftodio  -  TV  Announcer 

Accountant 

2 

Sales  Promotion 

Farmer 

1 

Pol  itician 

BOTTOM 


BOTTOM 


Intercorrelation  Matrix  —  AOC's*  During  Pre-Flight  Training 


Intercorrelation  Matrix  —  Cadets*  During  Pre-Flight  Training 
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Intercorrelation  Matrix  —  AOC's*  After  Pre-Flight  Training 
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Intercorrelation  Matrix  —  Codet**  After  Pre-Flight  Training 
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